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EXECUTIVE SUMMARY 


This report presents the findings of an experiment that investigated the effects of cultural 
background, message information content, rank status, and source credibility on situation assessment. 
Although information content, rank status, and source credibihty have received much attention by 
researchers in command and control decision-making, cultural variations in these factors have seldom 
been studied. 

Study participants were junior military officers from the U.S. and five foreign countries: Greece, 
Malaysia, the Philippines, Singapore, and Turkey. Messages describing ongoing events in an 
imaginary geopolitical region provided the decision-making context. At the outset, each participant 
received instmctions that either (1) favored an attack by one country on another country in that 
region, (2) favored the occurrence of war games in that country, or (3) that were neutral with regard 
to the Ukehhood of attack and war games. The 100 messages were organized into 10 blocks of 10 
messages. Each message originated from one of three sources that differed in terms of their 
credibihty. After reviewing each block of messages, participants estimated the hkehhood of attack 
and rated their confidence in that estimate. To examine effects of rank status, simulated e-mail 
messages were inteijected into the message stream at periodic intervals. These messages requested 
participants to assess a situation that was not directly related to the primary task. Responses provided 
from superior and subordinate sources either agreed or disagreed with the participant’s original 
assessment, which could then be changed if the participant so desired. 

All cultural groups increased their estimates of the hkehhood of attack as more messages about the 
developing situation were reviewed. However, the groups differed significantly in the rates at which 
their estimates increased. Participants from Malaysia, the Phihppines, and Singapore initially made 
lower estimates, but then increased to a level comparable with the other participants as the study 
progressed. Participants were most influenced by the messages they received in the first half of the 
experiment regardless of whether those messages came fi'om a high- or low-credibihty source. 
Participants had a strong tendency to assess the situation as soon as possible after encountering it, 
regardless of the quality of information used to make those assessments. The rank status manipulation 
found that, across cultural groups, the most common strategy was to change the initial assessment 
only when both superior and subordinate dissented. 

Study findings contribute to understanding the effects of cultural variables on situation 
assessment. These findings can be used to enhance the development of a database that relates cultural 
characteristics to decision-making preferences and tendencies. Future research should build on these 
findings by investigating in detail the relationship between cultural attributes, organizational factors, 
decision biases, and situation assessment variables. 
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INTRODUCTION 


This report presents the findings of an experiment that investigated the effect of cultural back¬ 
ground and selected decision variables on situation assessment. The decision variables included 
source credibiUty, rank status of information sources, and the degree of correspondence of one’s own 
with other’s judgments. Although researchers have closely studied these decision variables in 
Command and Control (C^) decision-making, cultural differences in situation assessment have 
received little attention. 

BACKGROUND 

Understanding cultural differences is vital in a world where there is ever-increasing contact iunong 
people from diverse cultural groups. For example, the use of multi-national coalition forces in both 
war iuid peacetime operations emphasizes the importance of considering the cultural differences that 
exist among coalition forces. Failure to do so can hamper collaborative decision-making and seriously 
compromise operation outcomes. Research on decision heuristics and situation assessment will be 
summarized here to provide a context for the present study. 

Cultural Values 

Literature reviews completed earlier in the Cognitive Aspects of Decision-Making (CADM) 
project established that values play a central role in differentiating cultural groups. Five cultural 
values were identified that vary from culture to culture: individualism-collectivism, uncertainty 
avoidance, power distance, masculinity-femininity, and Confucian work dynamism. Research by 
Hofstede (1980,1983, 1991), the Chinese Culture Connection (1987), and Schwartz (1994) converge 
to the view that these values influence decision-making processes and outcomes. The fact that 
independent researchers using different methods to measure culture obtained similar results indicates 
that these cultural values exert widespread influence on the lives of many people. In the experiment 
reported here, two of these values. Power Distance and Uncertainty Avoidance, were the basis for 
hypotheses for a situation assessment decision task. 

Uncertainty Avoidance is the extent to which planning and stabiUty are used as strategies in 
response to uncertainty. In a high Uncertainty Avoidance culture, people who are considered 
experts are given the most authority, explicit role differentiation is preferred in group interaction, 
adherence to mles is valued, there is considerable resistance to change, and a conservative 
approach is taken when dealing with ambiguous situations. Low Uncertainty Avoidance is 
generally associated with the corollary conditions outlined above, including higher levels of risk¬ 
taking (Cummings and Worley, 1994). 

Power Distance is the amount of deference shown by those in subordinate positions towards 
those in superior positions in an organization. High Power Distance is associated with activities 
and attitudes such as autocratic decision-making, superiors considering subordinates as part of a 
different class, close supervision of subordinates, and subordinates not Ukely to disagree with 
superior’s decisions. Low Power Distance is associated with egalitarian procedures (Cununings 
and Worley, 1994). 

These two values were emphasized in this study because of their relationships with organizational 
factors and decision-making variables. This experiment investigated how these values could be used 
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to differentiate and describe the decision performance of people belonging to different cultural 
groups. 

Decision Heuristics 

Decision heuristics are information processing strategies that are often used to simplify real-world 
decision situations. Decision heuristics enable people to process information that is complex, 
uncertain, or ambiguous to reach an appropriate decision. On many occasions, they facilitate 
effective action that would otherwise require considerable additional time and effort. Sometimes,, 
however, decision heuristics do not produce an effective response. Instead, decision heuristics result 
in systematic judgmental biases that produce inappropriate actions and prevent decision-makers from 
achieving a desired goal. 

Research on decision heuristics has been conducted mostly in North America and Western Europe. 
Because of this narrow cultural focus, information is limited about whether and in what manner 
cultural factors moderate decision heuristics. When interacting with people from different cultures, a 
thorough understanding of cultural factors, decision heuristics, and how they interact is prerequisite 
to achieving consistent and predictable outcomes. 

The present experiment represents an important first step in defining the relationship between 
cultural factors and decision heuristics. It examines the anchoring and adjustment decision heuristic 
in the context of a situation assessment task. Anchoring and adjustment is a pervasive phenomenon 
wherein the starting point for some situation cuichors later judgements. As more information becomes 
available about that situation, revised judgments fail to give enough weight to this later information. 
Consequently, judgments are biased toward the starting point. 

The practical imphcations of the anchoring and adjustment bias for decision-making are 
numerous. Information provided during the earliest stages of a developing situation is a crucial 
determinant of responses to later information. Biases favoring a certain decision outcome can be 
established by controlUng the content of the information that is presented early in a decision situation. 
Because adjustments from the initial anchor are typically overly conservative, predictions can be 
made regarding intermediate and final situation assessments when given knowledge of the 
information supplied after the initial information. 

In addition to decision heuristics, many other factors influence the manner in which situation 
assessment occurs and the outcome of the assessment process. This experiment examined source 
credibility and organizational factors, such as rank status, in a cross-cultural context. These variables 
have been studied using U.S. participants, but few documented studies have been conducted using 
participants from other cultures. 

Source Credibility 

The credibility of information sources is often an important factor in determining whether 
information is believed and acted upon. Indeed, source credibility is more important than source 
expertise in determining the importance of information. Limited information from a source perceived 
as highly credible is given more weight than a predominance of contradictory information from less- 
credible sources (O’Reilly and Roberts, 1976). 

Research on the representativeness decision heuristic has contributed knowledge that aids 
understanding the role of source credibility in situation assessment sources (Kahneman and Tversky, 
1972). According to the representativeness heuristic, the Ukelihood of an event is estimated by the 
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extent to which it is similar to its parent population. If properly applied, this heuristic facilitates 
effective decision-making. However, people sometimes fail to consider factors that are representative 
of a population when forming likelihood estimates, instead rel 5 dng on more salient, but less valid 
factors. For example, information sources that provide highly variable information tend to be less 
reUable than sources that provide information that is more consistent. However, this variabihty tends 
to make the source more salient to the observer, who then places undue weight on the information 
conveyed by that source. In this manner, lower credibility sources are relied on to a greater extent 
than higher credibility sources. 

Predictions also tend to be consistent with the most recent evidence, and rare events are often 
overemphasized if they are typical of available evidence. Research indicates that reliance on extreme 
cases and available information (both of which can be non-representative) are used to justify 
decisions. Statistically, however, data extremity emd availabiUty are often inversely related to 
assessment accuracy. Consequently, people tend to be overly confident in poor judgments (Kahneman 
and Tversky, 1973). 

The reliability of a source should be considered when assessing its credibility. However, in many 
situations, it is either not possible or decision-makers fail to consider source reliability. The present 
study investigated the role of source credibility in a situation assessment task. Each participant was given 
explicit instmctions before starting the experiment about the reliability of each of three source classes. 

One major interest of this experiment was determining whether participants considered source reliability 
when assessing the situation. 

Organizational Factors 

The effect of the social position of a source within an organization is a reliable predictor for 
whether or not its co mmuni cations are accepted. The present experiment examined rank status issues 
cross-culturally to determine the influence of cultural values such as power distance and uncertainty 
avoidance on responses to rank status during a decision task. In a study that investigated rank status 
influences among U.S. miUtary personnel, Driskell and Salas (1991) found that subordinates were 
more likely to defer judgment to superiors, and across ranks, highly stressed team members were 
more deferential. As an example, consider Power Distance, the amount of deference between 
superiors and subordinates in an organization. We can be hypothesize that a decision-maker from a 
low Power Distance culture will be less likely to defer to a contradictory judgment originating from a 
superior than a decision-maker from a high Power Distance culture. Other relationships among rank 
status and message credibility will also be investigated. Certain cultures depend more on the remk 
status of individuals and groups involved in decision-making than do others (McMillan, Hickson, 
Hinings, and Schneck, 1973). The U.S. and Canada, for example, tend to have more specialized rank 
status relationships than does Great Britain. This increased specialization guides decision-making 
autonomy and establishes the range of authority that people in those roles can exercise. This 
experiment examined the role of rank status in certain Asian and European cultural groups in its 
influence on decision-making. 

PURPOSE 

This experiment examined the relationship between selected properties of information, cultural 
variables, and measures of situation assessment. Surprisingly little research on command and control 
decision-making has explored how cultural factors influence the way that information is perceived 
and used in d 5 mamic decision-making situations. 
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SCOPE 


This experiment investigated the influence of specific cultural variables, source credibility, critical 
message content, and rank status on decision-making in a laboratory simulation “micro-world” task. 
Interest centered on determining whether the decision variables were related to differences in 
situation assessment by people with different cultural backgrounds. 

OBJECTIVES 

The principal objectives of this experiment included the following: 

• Describe how people from different cultural backgrounds respond to ambiguous and 
conflicting information in order to assess a developing situation 

• Determine how selected cultural, cognitive, and organizational variables influence situation 
assessment 

ORGANIZATION OF THIS REPORT 

The Methods section describes the materials and procedures used to measure situation assessment 
and cultural background. The Findings section presents the results of the experiment as cultural 
factors, source credibility, critical message content, and rank status. The Conclusions and 
Recommendations section discusses the implications of the findings for future research in the CADM 
project. 
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METHODS 


This section describes participant recruitment, the demographic and cultural questionnaire; the 
development of the experimental scenario, and the procedures used to conduct the experiment. 

PARTICIPANTS 

Participants were recmited from the student population at the Naval Postgraduate School in 
Monterey, California. Because a major aim of the CADM project is to examine cultural influences on 
decision-making, participants were drawn from several countries to obtain a culturally diverse 
sample. All but one participant was a military officer from the U.S. or one of five foreign countries: 
Greece, Malaysia, the Philippines, Singapore, and Turkey. The lone exception was a civilian 
employee of the Malaysian military. Three Hofstede cultural clusters were represented by the six 
countries included in this study: Cluster 4 (Greece and Turkey), Cluster 7 (Malaysia, Philippines, and 
Singapore), and Cluster 9 (U.S.). Approximately the same number of participants from each cultural 
group was recruited. All participants took part in the experiment as part of their daily work 
assignment and received no additional compensation for their participation. 

DEMOGRAPHIC AND CULTURAL QUESTIONNAIRE 

To characterize each participant as demographic and cultural factors, a two-part questionnaire was 
administered before the experiment. The first section of the questionnaire obtained demographic 
information including gender, age, education, occupation, citizenship, language typically spoken in 
various settings, style of food eaten, and religious background. We used responses to this section to 
summarize participant demographic characteristics and categorize participants into cultural groups 
based on markers of enculturation into a cultural group. The second section of the questionnaire 
assessed five cultural value dimensions that have been extensively used by researchers to investigate 
cross-cultural issues. Hofstede (1980, 1983) has identified four of these dimensions: 

Individualism-Collectivism : the degree to which one’s identity is defined by personal choices 
and achievements or by the character of the collective groups to which one is more or less 
permanently attached. 

Uncertainty Avoidance : the extent to which one focuses on planning and stability as ways to 
respond to life’s uncertainties. 

Power Distance : the amount of respect and deference between those in superior and 
subordinate positions in an organization or group. 

Masculinity-Femininity : the relative emphasis on achievement and interpersonal harmony 
that characterizes gender differences in some national cultures. 

A fifth cultural value dimension, Confucian Work Dynamism, further differentiated variations in 
cultural background. This dimension assesses the extent to which a person adopts a long-term 
orientation toward life (Chinese Culture Connection, 1987). Scores on the four Hofstede dimensions 
were calculated using the methods described in Hofstede (1980). Confucian Work Dynamism scores 
were computed as specified by the Chinese Culture Connection (1987). Together, these five cultural 
dimensions characterize individuals as culturally based differences in values and behavior patterns 
that help explain decision-making based on uncertain and ambiguous information. 
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SCENARIO DEVELOPMENT 


A computer-based “micro-world” simulation describing a water rights conflict was developed by a 
subject matter expert familiar with military decision-making in situations involving incomplete, 
uncertain, and ambiguous information. The scenario described an imaginary geopolitical setting in 
which several neighboring countries had unequal access to limited water supplies. The possibility was 
raised that one of the countries might stage border incursions over disputed territory containing water 
resources. These countries differed in military capabihties, social and political circumstances, 
geographic features, and natural resources, among other factors. 

The scenario permitted two mutually exclusive events: war games in one of the countries or an 
attack by that country on another country. The scenario was written to make either of these events 
viable. Participants were told that they were to monitor incoming message traffic and provide 
periodic situation assessments regarding the likelihood of an attack by one country on another 
country. 

The scenario was implemented using a multimedia programming application, Macromedia 
Director, and run on Macintosh computers. A color monitor displayed the experiment to the 
participant and was the response interface. Figure 1 shows an example of the display as the 
participant viewed it during the experiment. A map occupying the upper part of the screen showed 
the countries of interest, military units of other countries (green squares), major transportation 
elements (railroad tracks, airports, bridges), geographic features (rivers, lakes, drainage areas), and 
country boundaries and names. A flashing yellow circle highlighted the area referenced by each 
message. A legend for the map symbols was located to the right of the map. Several items were 
displayed below the map, including each new message, its source, when it was sent, and the elapsed ■ 
time since the prior message. 

PROCEDURE 

A set of 100 messages, organized into 10 blocks of 10 messages each, described events in an 
imaginary geopolitical region. Based on these messages, participants estimated the hkelihood of an 
attack by one country on another country in that region. There were three types of messages in each 
block of 10 messages. Six of the messages were intended to be non-critical, with no direct relevance 
to estimating the likehhood of an attack. Of the remaining four messages, two were written to support 
the idea that an attack was likely to occur and two were written to support the idea that war games 
were more likely. Messages were presented singly and participants moved through each block of 
messages at their own pace. 

Upon completing each block of messages, a screen hke that in figure 2 was displayed. Participants 
entered a likelihood estimate for attack using a seven-point scale that ranged from “Very Low” to 
“Very High.” A confidence rating was also made for the likelihood estimate using a 10-point scale. 
The 10 messages from the preceding block were displayed on the screen along with their sources 
while participants entered their updated hkelihood estimates and confidence ratings. Participants also 
indicated which of the 10 messages were most relevant in making their hkelihood estimate and 
confidence rating. Chcking on the “DONE” button after making their responses moved the 
experiment to the next block of 10 messages. 
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Figure 2. Sample screen during block assessment period. 


7 
























^yvnhnl ie:<*y , 



Figure 3. Sample screen showing simulated e-mail message and response options. 


Source credibility was examined by assigning one of three levels of credibility to each message. 
The six non-critical messages in each block were always assigned to open sources that were described 
as newspapers, magazines, national/international television and radio, and international organizations. 
Participants were told that open source information could normally be accepted. The four critical 
messages in each block were assigned to either established covert sources or new covert sources. The 
instructions provided before beginning the experiment described each source in terms of its 
reliability, depicting the established covert source as more reliable and accurate than the new covert 
source. 

The influence of rank status on decision-making was investigated by interjecting a simulated 
e-mail message at periodic intervals into a block of messages. The e-mail message requested that 
participants judge whether a certain event would occur. After making the initial assessment, 
participants forwarded the message along with their assessment to a superior and a subordinate for 
review and evaluation. After a short time, one of four reply combinations was displayed on the 
screen: (1) the superior and subordinate agreed with the participant’s assessment, (2) the superior and 
subordinate disagreed with it, (3) the superior agreed, but the subordinate disagreed, and (4) the 
superior disagreed, but the subordinate agreed. Participants were allowed to revise their assessment 
according to this response before submitting their final recommendation. Figure 3 shows a sample 
screen for the rank status manipulation. 

Each participant was tested individually. The participant was seated in front of a desktop computer 
equipped with 17-inch color monitor. The experimenter explained the purpose of the study and the 
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participant was asked to sign an informed consent statement if agreeing to participate. The 
demographic and cultural background questionnaire (Appendix A) was then administered. Upon 
completing this questionnaire, the participant read a summary of the scenario background and an 
update of the current situation in the region of interest (Appendix B). 

Next, the experimenter trained participants in using the computer to enter responses to each 
message. The instmctions presented to the participants are contained in Appendix C. A practice 
session allowed participants to fa mili arize themselves with scenario features and response entry 
procedures. Participants were told to play the role of an analyst of one country, “Alantra.” In this 
role, they would monitor and evaluate message traffic pertaining to actions by a country of interest in 
the region of concern. In this capacity, participants estimated the relevance of each message for 
predicting the occurrence of an attack upon Alantra by a neighboring country, Ngolo. 

Participants were told that several blocks of messages would be presented and that after each block 
of messages they would be asked to estimate the likelihood of an attack, to rate their confidence for 
that estimate, and to indicate which of the 10 messages from the preceding block were the strongest 
determinants of their responses. The final instructions and information update were presented after 
the practice session, but before starting. The experimental trials are contained in Appendixes D and 
E, respectively. There were three versions of the final information update, depending on the anchor 
condition to which each participant was assigned: attack, war games, or neutral. 

According to the anchoring and adjustment bias, information encountered early in a situation 
assessment is often more influential on decisions than later information. To examine the effects of 
this bias, three instruction sets were prepared that described different probable courses of actions by 
Ngolo. These instmction sets were presented to participants just before they began the actual 
experiment trials to gain the maximal anchoring effect. For the attack instruction set, statements were 
provided that supported the occurrence of an attack over some other alternative, including war games. 
An analogous approach was used in the war games instruction set. The neutral instmction set 
contained statements that showed attack and war games as equiprobable events. 
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FINDINGS 


This experiment exeimined cultural variations in situation assessment as influenced by the initial 
information provided about a situation, credibiUty of message sources, rank status of people who 
provided feedback, and correspondence with other’s judgments. 

PARTICIPANT DEMOGRAPHIC AND CULTURAL CHARACTERISTICS 

Participants were 39 male students attending the Naval Postgraduate School in Monterey, California 
(mean age = 31 years). All but one were military officers for their home country; the lone exception was 
a civilian employee of the Malaysian mihtary. To aid in the categorization of participants into the 
appropriate cultural group, an enculturation index was computed for each participant. The enculturation 
index is composed of items measuring several demographic vmables. It ranges from 0 to 6, with 
increasing values indicating greater commonality with the home country (the country with which the 
participant identified). A low enculturation score for an individual indicates that an individual may not 
provide responses that are reflective of the home country. The enculturation index for each participjuit 
was highest for the home country each initially listed, requiring no changes to self-identified home 
country. Table 1 shows the distribution of the 39 study participants according to the 
a priori Hofstede culture clusters, the enculturation index, and home country. 


Table 1. Cultural charac 

teristics of participants. 

Hofstede 

Cluster 

n 

Mean Enculturation Index 
(Range) 

Home Country (Number of Participants) 

Cluster 4 

14 

5.67 (4.93-6.00) 

Greece (2), Turkey (12) 

Cluster 7 

13 

5.69 (4.94-6.00) 

Malaysia (3), the Philippines (2), 

Singapore (8) 

Cluster 9 

12 

5.93 (5.65-6.00) 

U.S. (12) 


For analysis, participants were grouped according to the cultural cluster profiles identified by 
Hofstede (Harzing and Hofstede, 1996). The profiles and cluster groupings in table 2 were identified 
from data of over 100,000 individuals in organizationally matched samples. These profiles provided a 
benchmark for comparing the participants in this experiment with their corresponding Hofstede 
culture cluster. These profile values are well-established in the literature and generally accepted by 
most cross-cultural researchers. Therefore, groupings of participants for analysis were based on the a 
priori placements of their home countries on these dimensions. Because of small sample size, 
groupings based on these a priori values eire more vahd for hypothesis evaluation. 
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Table 2. Hofstede cultural value profiles for clusters 4, 7, and 9. 


Hofstede 

Cluster 

Individualism- 

Collectivism 

Uncertainty 

Avoidance 

Power 

Distance 

Masculinity- 

Femininity 

Cluster 4 

mid 

high 

high 

mid 

Cluster 7 

low 

low 

high 

mid 

Cluster 9 

high 

low 

low 

high 


Table 3 shows the four Hofstede dimension values for the participants in this experiment. 
Comparison of these values with the table 2 benchmarks reveals a relatively low level of 
correspondence for all three culture clusters. The reasons for this lack of correspondence are not 
immediately clear. Perhaps the restricted nature of the sample in this study was a contributing factor. 
All participants were students in an U.S. military postgraduate school. Therefore, they may not have 
been highly representative of their home culture. Note that the original Hofstede study upon which 
the cultural value profiles in table 2 are based was conducted more than 30 years ago. Certain aspects 
of these cultural values might have changed. 



fable 3. Empirical cu 

tural value profiles for clusters 4, 7, and 9. 


Individualism- 

Collectivism 

Uncertainty 

Avoidance 

Power 

Distance 

Masculinity- 

Femininity 

Cluster 4 

mid 

mid 

high 

low 

Cluster 7 

mid 

mid 

low 

low 

Cluster 9 

mid 

mid 

mid 

mid 


CULTURAL VARIATIONS IN LIKELIHOOD ESTIMATES 

Figure 4 shows the attack likelihood estimates for each culture cluster for all block assessments. The 
two-way interaction of culture by assessment period was significant (F (7464 9 ^ 7 ) = 2.15, p < .05, 

r|^= .125). All three cultural groups increased their estimate of the likelihood of attack as they were 
provided with more information about the developing situation. However, the groups differed in their 
rates of increase over time. 

The initial assessment (Bloek 1) of Clusters 4 and 9 was the same, and Cluster 7 was lower, but by 
the final assessment (Bloek 11) Clusters 4 and 9 had the most dispcU’ate values, with Cluster 7 in the 
middle. The relative positions of the clusters changed several times over the 11 assessment blocks. 
These differences could be related to cultural differences in situation assessment, the details of which 
could be examined by future studies that focus on the interaction of cognitive and environmental 
variables in deeision-making situations. 
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Figure 4. Attack likelihood estimates by culture cluster. 

CULTURAL VARIATIONS IN CONFIDENCE 

Figure 5 shows how the confidence ratings of each culture cluster varied as the experiment 
progressed. As with the likelihood ratings, there was evidence for a culture by assessment period 
interaction, although after adjusting for a violation of sphericity, it just failed to attain significance 
(p = .059) (p < .05 without the adjustment). Culture cluster 7 (Malaysia, the Philippines, Singapore) 
differed from the other two clusters, starting lower and following a less-predictable pattern as the study 
progressed. However, all three clusters became more confident in their likelihood estimates as more 
information was available. 



Assessment Period 

Figure 5. Confidence ratings by culture cluster. 
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ANCHORING 


The anchor manipulation presented each participant with instructions, seemingly the report of a 
former watchstander. These instructions favored either an attack by one country on another country in 
that region (attack anchor) or the occurrence of war games in that country (war games anchor). 
Another condition in which instructions were neutral was concerning the likelihood of attack, and 
war games were designed as a control. We hypothesized that likelihood assessments for attack would 
be higher from participants presented with the attack anchor than from participants presented with the 
war games anchor. Figure 6 shows the mean likelihood estimates of participants in each anchor 
condition at initial assessment and final assessment. Although the pattern of means was in the 
expected direction, a statistically significant anchoring effect was not obtained. Because of the 
absence of an anchoring effect, investigation of sufficiency of adjustments from the initial anchor 
position was not attempted. 
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Figure 6. Mean likelihood estimate for attack by anchor condition. 


IDENTIFICATION OF CRITICAL MESSAGES 

For each block of 10 messages, participants identified up to five messages that they considered 
most important in making their likelihood estimate. The messages identified by the participant were 
compared with the set of critical messages to determine how well they matched. Figure 7 shows that 
the culture clusters varied significantly in the frequency with which critical messages were identified 
(F (2, 36) = 3.667, p < .05). Post-hoc tests (Tukey RSD, p < .05) found that participants from the U.S. 
identified more critical messages per block than the group from Greece and Turkey (72% versus 58% 
mean critical messages identified per block). Neither the U.S. nor the Greece/Turkey groups differed 
significantly from the group from Malaysia, the Philippines, and Singapore (65% of the critical 
messages identified per block). This pattern might be at least partially attributable to differences in 
fluency with the English language. The demographic data show that as a group, participants from 
Greece and Turkey (cluster 4) had the least experience, and the participants from Malaysia, the 
Philippines, and Singapore had considerably more experience, although less than the U.S. 
participants. These differences in English language experience might have influenced the ability of 
participants to read and properly interpret the critical messages. 
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(Cluster 4) Singapore (Cluster 7) 


Figure 7. Mean percentage of critical message correctly identified per block. 


Figure 8 further decomposes the results in figure 7 into the mean percentages of the attack-related 
and war games-related critical messages that each culture group correctly identified per block. Figure 
8 includes only the participants who received either the attack or the war games anchor (whereas 
figure 7 includes all participants, including those who received the neutral anchor). All groups 
identified a higher percentage of the attack-related messages than the war games-related messages. 
Inspection of these two classes of messages revealed that the attack-related messages tended to appear 
more salient than the war games-related messages. The attack-related messages usually made a direct 
statement pertaining to an attack or made a strong implication that an attack was likely. The war 
games-related messages tended to be subtler, often requiring some interpretation before an 
appropriate implication could made. 
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SOURCE CREDIBILITY 


Source credibility was examined by varying the type of source that delivered critical messages. In 
one condition, critical messages favoring an attack interpretation (pro-attack) were presented by a 
higher credibility source in the first half of the experiment and by a lower credibility source in the 
second half. In the other condition, the order was reversed. The source credibility manipulation 
enabled any differential influence of a higher versus a lower credibility source on likelihood estimates 
for attack to be examined. We hypothesized that source credibility would interact with the assessment 
periods such that attack messages from higher credibility sources would encourage higher likelihood 
estimates than attack messages from lower credibility sources. 

Figure 9 shows a significant interaction of source credibility by block for the likelihood estimate 
for attack, p < .05. Likelihood estimates increased in both halves, but more so during the first half 
(when a high credibility source provided critical messages) than in the second half (when a low 
credibility source provided critical messages). Note, however, that the likelihood estimates in the 
second half were higher than those in the first half, indicating that a low-credibility source not only 
did not erode judgments, but further enhanced them, although at a slower rate of increase. 

Figure 10 shows that the same general pattern emerged when lower credibility sources were 
initially used to present critical messages. In this case, there were two significant main effects for 
source credibility and block (p < .05), but no higher order interaction. Likelihood estimates in the first 
half increased at a greater rate than they did in the second half, and likelihood estimates in the second 
half were higher than in the first half. Evidently, higher rates of change occur in the early stages of 
assessment, regardless the level of source credibility. The fact that the critical messages in the first 
half were conveyed by low-credibility sources did not discourage using that information to assess the 
situation. 

These findings are noteworthy because they show that participants were most influenced by 
information they received in the first half of the study, regardless of whether it came from a high or 
low credibility source. The early stages of situation assessment seem the most influential in 
generating an assessment, regardless the credibility of the source. This tendency to learn about 
situations as soon as possible after encountering them might also influence how much subsequent 
information is weighted. 

Note that participant comments during the post-experiment debriefing regarding how they viewed 
source credibility did not match these findings. Most participants claimed that the message source 
was either moderately or decidedly important in determining their likelihood estimates for attack. 
These claims did not vary significantly by culture cluster. In addition, participants overwhelmingly 
chose the high-credibility source as the one they trusted the most. However, the findings reported in 
this section indicate that the need to learn about a situation took precedence over the credibility of the 
message source. Increases in likelihood estimates were greatest in the first half of the experiment 
regardless of whether the messages originated from a high or low credibility source. There seems to 
be a self-imposed need to assess a situation early even if that assessment is based on messages 
originating from a source that is not considered highly reliable or accurate. 
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Figure 9. Likelihood estimates for attack when high-credibility sources presented pro-attack 
critical messages in first half of experiment. 



Figure 10. Likelihood estimates for attack when low-credibility sources presented 
pro-attack critical messages in first half of experiment. 




RANK STATUS 

The four e-mail messages interjected during the experiment enabled rank status issues to be 
addressed as Hofstede cultural values. Recall that after making the initial evaluation requested by an 
e-mail message, participants forwarded the message and their evaluation to a senior analyst 
(described to the participants as of superior rank status) and a Junior analyst (described as of 
subordinate rank status). For each of the four e-mail messages, responses from these two sources 
varied in the extent to which the senior and junior analyst agreed with the participant and with each 
other; (1) both analysts agreed with each other and with the participant, (2) both analysts agreed with 
each other, but disagreed with the participant, (3) the senior analyst disagreed and the junior analyst 
agreed with the participant, and (4) the senior analyst agreed and the junior analyst disagreed with the 
participant. 

A basic question was whether participants would change their evaluations in a predictable way 
with respect to the superior and subordinate rank status of the analysts who responded for their 
evaluations. Note that no additional information or rationale was provided as part of the re.sponse, so 
influence was isolated from other factors in the decision environment. An analysis conducted on the 
percentage of participants who changed their initial evaluations after receiving respon.ses was 
statistically significant (Cochran’s Q (3) = 48.78, p < .05). Figure 11 shows that, as expected, the 
fewest changes occurred when both the superior and subordinate agreed with the participant, and that 
the most changes occurred when both of them disagreed with the participant. These results were 
expected based on a simple “majority rule” model of decision-making. In addition, this “majority 
rule” strategy was expounded by a preponderance of participants who changed their initial 
assessments; this strategy did not vary significantly by culture cluster. The two mixed conditions 
(superior and subordinate disagree) generated results that more directly addressed rank status issues. 
Pairwise comparisons (McNemar Test) of each condition with the condition where both superior and 
subordinate disagreed showed that significantly fewer changes occurred in the former conditions. 
Although approximately three times more evaluations were changed when the superior rather than the 
subordinate disagreed with the participant, this difference was not statistically significant. 
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Both Agree Superior Agree- Superior Both Disagree 
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Disagree Subordinate 
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Superior-Subordinate Agreement with Participant 

Figure 11. Percentage of participants who changed their evaluations by the four 
superior-subordinate agreement-disagreement combinations. 


Power Distance 

We hypothesized that when the dissenter was a superior, participants from high Power Distance 
cultures (clusters 4 and 7) would change their evaluations more frequently than participants from a 
low Power Distance culture (cluster 9). The rationale for this prediction was that people in high 
Power Distance cultures have been trained as more deferential toward those in superior positions in 
an organization than people in low Power Distance cultures. Figure 12 shows that the predicted 
pattern of results was obtained, although the difference between the two groups was not statistically 
significant. More than 26% of participants from high Power Distance cultures (clusters 4 and 7) 
changed their initial evaluations when the superior disagreed with it, compared to just 8% of the 
participants from the low Power Distance culture (cluster 9). 

Figure 13 shows the percentage of changed evaluations when the subordinate rather than the 
superior disagreed. We hypothesized that participants from a low Power Distance would change their 
evaluations more frequently than participants from a high Power Distance culture. The rationale for 
this prediction was that people from low Power Distance cultures, having a more egalitarian 
perspective, would tend to accept advice from subordinates more readily than people from high 
Power Distance cultures. However, participants from high Power Distance cultures changed their 
evaluations more often than participants from the low Power Distance culture, although this 
difference was not statistically significant. The percentages were much lower in both cases. No 
participants from the low Power Distance culture, and only 10% of those from the high Power 
Distance cultures, changed their evaluations when the subordinate disagreed with them. 
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Figure 12. Percentage of changed evaluations by high and low 
Power Distance cultures when only the superior disagreed. 
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Figure 13. Percentage of changed evaluations by high and low Power Distance 
cultures when only the subordinate disagreed. 


Uncertainty Avoidance 

Uncertainty Avoidance was another Hofstede cultural value that was discussed in the context of 
rank status. Participants from a high Uncertainty Avoidance culture (cluster 4) were hypothesized to 
change their evaluations more often than participants from low uncertainty avoidance cultures 
(clusters 7 and 9). The rationale for this prediction was that uncertainty instilled by disagreement of a 
co-worker would be reduced by changing one’s decision to be consistent with the other. Figure 14 
shows the findings which, while not statistically significant, exhibited the hypothesized trend. 
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CONCLUSIONS 


CULTURAL CHARACTERISTICS 

Only a modest correspondence was found between our sample’s cultural dimension profiles and 
those of Hofstede and the Chinese Cultural Connection. Our sample was a different and very 
restricted one in organizational membership. All were employees of their respective countries’ 
militaries, and not, therefore, necessarily representative of their countrymen. Compounding this and 
representing a major challenge is the fact that all foreign participimts are currently residing in the 
U.S. and using a language that is not their native tongue. Finally, the sample size is limited. In prior 
research with a larger sample (Heacox, Gwynne, Sander, and Fleming, 1998), a higher level of 
correspondence was obtained. Hofstede and the Chinese Culture Connection have cautioned that the 
validity of these dimensions relies on large samples. Therefore, the a priori profiles of these research¬ 
ers were used for analysis of hypotheses related to culture cluster. 

LIKELIHOOD ESTIMATES 

Although all three culture groups increased their likelihood assessments as the study progressed, 
they did so at different rates. Participants from Malaysia, the Philippines, £md Singapore (cluster 7) 
tended to vary from the other two groups who differ less from each other. The largest differences 
occurred at the outset of the assessment task when the cluster 7 participants made more conservative 
(lower likelihood) estimates. By the third period, however, they had increased to approximately the 
same level as clusters 7 and 9. 

ANCHORING AND ADJUSTMENT 

The failure to obtain a statistically significant anchoring effect made the interpretation of that 
portion of the experiment problematic. Despite the lack of statistical evidence for an anchor, the 
anticipated pattern of anchoring effects was obtained. Participants who received the attack anchor 
made the highest initial likelihood estimates for attack, and those who received the war games anchor 
ma de the next highest estimates, followed by those who received the neutral anchor. This trend 
suggests that the anchor manipulation had its intended effect, although it failed to attain a sufficient 
magnitude to reach statistical significance. The power analysis conducted during the design of the 
experiment assumed that the anchor manipulation would produce a moderate effect size. Evidently, 
the effect size was smaller, with the result that too few participants were included to provide the 
power needed for statistical significemce. 

An additional issue should be discussed concerning anchoring and adjustment. Given the context 
of this experiment, what constituted a sufficient adjustment or an insufficient adjustment from the 
anchor? Anchoring and adjustment experiments typically use a quantitative anchor to facilitate its 
measurement and gauging movement from it. This experiment did not incorporate an explicit metric 
for measuring an anchor and movement from it. Because of this, subsequent movement from the 
initial imchor was difficult to evaluate in terms of sufficiency of adjustment. However, if a significant 
anchoring effect were obtained, the adjusted likelihood estimates that ^lre made following a neutral 
anchor could be considered as “sufficient.” Adjustments made in other anchor conditions could then 
be compared to the control (neutral) anchor condition. The fact that all three conditions increased as 
the experiment progressed indicates that there was a general tendency to perceive an attack as 
becoming more likely regardless of anchor. 
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CRITICAL MESSAGE IDENTIFICATION 


The unintended differential saliency of the attack and war games critical messages complicates 
interpretation of the attack likelihood estimates. Recall that in each block of 10 messages there were 
four critical messages and six “background” messages. The critical messages were evenly divided 
into two attack messages and two war games messages. The rationale for this design was that 
participants who received the attack anchor would mcike higher estimates for attack than participants 
who received either the war games anchor or the neutral anchor, because even though balanced 
amounts of information was presented, those who had received an attack anchor would find those 
messages more salient. However, participants with both attack and war-game anchors found the 
attack messages more sahent. Analysis of the messages that participants identified as most relevant 
for making the attack likelihood estimates revealed that attack messages were identified most of the 
time. (A different story for the war games critical messages, however; they were missed considerably 
more often). 

SOURCE CREDIBILITY 

The major source credibility finding was that likelihood estimates increased more in the first half 
of the experiment than in the second half, regardless of whether higher or lower credibility sources 
were supplying the critical messages. A straightforward interpretation of this finding is that 
participants’ need to generate a situation assessment outweighed consideration of the accuracy and 
reliability of the information upon which that assessment was based. A very interesting observation, 
noted earUer, is the discrepancy between these findings and participants’ verbal reports of the 
importance of the source. The finding that information from acknowledged low credibility sources 
has implications for information operations. Information from questionable sources may have more 
influence power than previously assumed, especially if this information occurs early in a situation 
assessment task. 

RANK STATUS 

The major rank status finding was that all cultural groups employed a “majority rules” decision 
rule. The sheer number of dissenting opinions was the major determinant in whether a participant 
changed his initial assessment. Recall that no further information was provided by the dissenters-they 
merely disagreed. Several participants, when debriefed, said, “when both disagreed I figured they 
knew something that I didn’t know.” 

Driskell and Salas (1991) placed U.S. military participants in a situation in which they reviewed 
several two-pattem checkerboard slides and made an initial choice of which of the two patterns 
contained more white space (in reality there was no difference). When participants received a 
dissenting opinion from a “team member” lower in status, they changed their initial estimate 34% of 
the time. When the dissenting opinion came from one higher in status, participants changed 51% of 
the time. Our results cannot be directly compared in part because there were two other members 
providing input, therefore adding a majority-minority within-group dynamic. In addition, the rank 
status condition Driskell and Salas studied was between-subjects rather than within-subjects. 
However, in general, a pattern eonsistent with their findings was observed; there were fewer changes 
when only subordinates disagreed, and more when only superiors disagreed. 

Our findings show the power to influence decisions that exists within a group, espeeially in 
ambiguous situations. A stated majority position might result in a change of decision by a minority, a 
change that is not necessarily warranted. The hypotheses related to cultural dimensions were not 


24 



supported, although two of the three findings were consistent with the hypotheses. The predictive 
power of these broad cultural dimensions appears to be compromised with small sample sizes. 

The hypothesized cross-cultural differences in Power Distance and Uncertainty Avoidance in 
terms of the influence of rank status on changing evaluations were not obtained. Military membership 
might exert a stronger influence than broader cultural values on task performance, especially when 
those tasks are highly specific to military organizations. 
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APPENDIX A 

DEMOGRAPHIC AND CULTURAL BACKGROUND QUESTIONNAIRE 



You are being asked to participate in a scientific study of decision-making. As joint 
international military and business ventures are occurring more frequently, it is 
important to understand how people with diverse backgroimds can work together to 
make good decisions. Toward this end, the current study is being conducted by the 
Research, Development, Test and Evaluation Division of the Space and Naval Warfare 
(SPAWAR) Systems Center San Diego (SSC San Diego). 

Yoiu- participation is voluntary. If you agree to participate, please fill out the attached 
survey. If you prefer not to participate, simply return this form to your professor or 
research assistant. 

This survey is anon)nnous. Do not write your name on the survey. 

If you have questions regarding the study, or if you would like to receive a summary 
report of survey findings, please contact one of the researchers: Dr. Steve Sander, 

SSC San Diego, telephone: 619-553-8006, or e-mail: sander@spawar.navy.mil, or 
Dr. Nancy Heacox, Pacific Science and Engineering Group, Inc., telephone: 619-535- 
1661, or e-mail: heacox@pacific-science.com. 

Your participation is appreciated. Please return completed surveys as directed to your 
professor or research assistant. 
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Background Survey 


Section A 


Are you active military? _Yes _No Are you _Male _Female 

What is your age? _ What is your religion? _ 

What is your current job or work assignment?_ 

What is your citizenship?_ 

What is the country with which you identify yourself?_ 

Note: In the following items, this coimtry is referred to as your "home country." 

How long have you lived in your home country? _years 

How long have you lived in another country? _years Country:_ 

(Please list additional years and countries on the back of this paper.) 

How often do you eat food prepared in the style of your home country? 

_Less than monthly _1 or 2 times a month _Weekly _Everyday 

Language: 

What language did you usually speak at home as a child?_ 

What language do you usually speak at home now? _ 

What language do you usually speak at school or work? _ 

What language do you usually speak with your friends? _ 

Please list the countries where you were educated: 

• Ages 6 -12: _ 

• Ages 13 -18: _ 

• CoUege/University/ 

Advanced Training _ 

If you are currently a student, what is your academic program of study? 

_Arts / Literature _Business _Education 

_Engineering _Life Sciences _Physical Sciences 

_Social Sciences Other (Ust)_ 







Section B; Please answer the questions in Section B relative to your current or 

MOST RECENT EMPLOYER IN YOUR HOME COUNTRY . IF YOU HAVE NEVER BEEN 
EMPLOYED IN YOUR HOME COUNTRY, PLEASE SKIP TO SECTION C. 


The descriptions below apply to four different types of managers. Read 

THROUGH THESE DESCRIPTIONS, THEN COMPLETE ITEMS #1 AND #2: 

Manager 1: Usually makes decisions promptly and communicates them to subordinates 
clearly and firmly. Expects them to carry out the decision loyally and without 
raising difficulties. 

Manager 2: Usually makes decisions promptly but before going ahead, tries to explain 
them fully to subordinates. Gives them the reason for the decision and 
answers whatever questions they may have. 

Manager 3: Usually consults with subordinates before reaching decisions. Listens to their 
advice, considers it, and then announces his/her decision. He/she then 
expects 

all to work loyally to implement it whether or not it is in accordance with the 
advice they gave. 

Manager 4: Usually calls a meeting of subordinates when there is an important decision 
to 

be made. Puts the problem before the group and tries to obtain consensus. 

If consensus is reached, the manager accepts this as the decision. If 
consensus is 

impossible, the mamager usually makes the decision him/herself. 


_ Mgr. 1 Mgr. 2 Mgr. 3 Mgr. 4 

#1. Regarding the above types of 

managers, please circle the one 12 3 4 

which you would prefer to work 

under. _ 

#2. Which one of the above types of 

managers is most like your own 12 3 4 

manager? _ 

Section B continues on the following page 
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Complete items #3 - #6 based on your experiences at your current or most recent 

EMPLOYER IN YOUR HOME COUNTRY: 


#3. 

Company rules should not be 

strongly 

disagree 

neutral 

agree 

strongly 


broken—even if the employee thinks 

disagree 

2 

3 

4 

agree 


it is in the company's best interest. 

1 




5 

#4. 

How frequently in your experience 

very 

seldom 

sometimes 

frequentl 

very 


does/did the following occur: 

seldom 

2 

3 

y 

frequently 


Employees being afraid to 
express disagreement with their 

1 



4 

5 


managers. 






#5. 

If currently employed: 







How long do you think you will 

2 years 

from 2 to 

more than 

until I 



continue working for this 

at most 

5 years 

5 years 

retire(d) h; 



company ? 

If not currently employed: 

1 

2 

3 

4 



How long did you work for your 
most recent employer? 






#6. 

How often do/did you feel nervous 

very 

seldom 

sometimes 

frequently 

very 


or tense at work? 

seldom 

1 

2 

3 

4 

frequently 

5 


Section Ci Please complete the items in Section C based on your personal 

PREFERENCES AND PRIORITIES. 


How important is each of the following to you personally: 




very 

xmimportant 

unimportant 

neutral 

important 

very 

important 

#1. 

Observation of rites and social 
rituals. 

1 

2 

3 

4 

5 

#2. 

Reciprocation (exchange) of 
greetings, favors and gifts. 

1 

2 

3 

4 

5 

#3. 

Ordering relationships by status 
and observing this order. 

1 

2 

3 

4 

5 

#4. 

Personal steadiness and stability. 

1 

2 

3 

4 

5 

#5. 

Thrift. 

1 

2 

3 

4 

5 

#6. 

Persistence (perseverance). 

1 

2 

3 

4 

5 

#7. 

Having a sense of shame. 

1 

2 

3 

4 

5 

#8. 

Protecting your "face" (i.e., 
maintaining your public image). 

1 

2 

3 

4 

5 















































































Section C (continued) 

How important is it to you to: 



very 

unimportant 

imimportant 

neutral 

important 

very 

important 

#9. Have challenging work to do— 
work from which you can get a 
personal sense of accomplishment? 

1 

2 

3 

4 

5 

#10. Live in an area that's desirable to 
you and your family? 

1 

2 

3 

4 

5 

#11. Have an opportunity for higher 
earnings? 

1 

2 

3 

4 

5 

#12. Work with people who cooperate 
well with one another? 

1 

2 

3 

4 

5 

#13. Have training opportunities (to 
improve your skills and to learn 
new skills)? 

1 

2 

3 

4 

5 

#14. Have good fringe benefits (paid 
vacation time, health insurance)? 

1 

2 

3 

4 

5 

#15. Get the recognition you deserve 
when you do a good job? 

1 

2 

3 

4 

5 

#16. Have good physical working 

conditions (good ventilation and 
lighting, adequate work space)? 

1 

2 

3 

4 

5 

#17. Have considerable freedom to 

adopt your own approach to a job? 

1 

2 

3 

4 

5 

#18. Have the security that you will be 
able to work for your company as 
long as you want to? 

1 

2 

3 

4 

5 

#19. Have an opportunity for 

advancement to higher level 
positions? 

1 

2 

3 

4 

5 

#20. Have a good working relationship 
with your manager? 

1 

2 

3 

4 

5 

#21. Fully use your skills and abilities 
on the job? 

1 

2 

3 

4 

5 

#22. Have a job which leaves you 

sufficient time for your personal 
and family life? 

1 

2 

3 

4 

5 


Thank you for completing this survey 
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Background and Current Situation 


Setting and Task: 

You have recently been assigned as a senior analyst for Ngolon affairs within the intelhgence 
directorate. You are supporting the Alantra poUtical leadership and various Alantra embassy 
officers throughout the region. You are being supported by other, more junior, analysts within 
the directorate. Your reports and analyses will normally be provided to your supervisor, the 
Chief of the Intelhgence Directorate. 

As part of your initial famiharization, you wiU be reviewing information about the background of 
the area, the current situation, and an analysis and situation assessment prepared by your immediate 
predecessor. This information is contained in the following pages. 

Background: 

The countries Alantra, Beyer, and Chardeaux have a long history as distinct nations. The historical 
separations, now borders among these nations, are based on ethnicity, tribal affihation and reUgion; 
plus the geographic features of the area. Beyer and Chardeaux share a common rehgion and 
heritage; their common border has changed from time to time, but rarely due to conflict. Rather the 
border changes have been to accommodate ancient tribal and famiUal affiUations. Alantra, on the 
other hand, is culturally unique. Its ethnicity and reUgion are quite different from the rest of the 
region. They have always been the most powerful nation in the region and look upon themselves as 
quite tolerant and benevolent. In the past they did exploit and at times persecute their neighbors. 
This era seems to have passed but the other nations in the region always critically examine Alantra’s 
motives as to alUances, negotiations, trade pacts, and other socio-economic initiatives. All three 
countries have developed a sophisticated political process that provides for direct representation. 

The nations each have a well-established election process. 

The nearby countries Ngolo and Esteban are relatively new nations and came into being during the 
post-colonial era by decree and the partition process. There are no natural or geographic features 
that define the boundary between Ngolo and Esteban and in the past only one country and people 
existed where there are now two. The pohtical process of both countries is relatively 
unsophisticated, with httle or no representation of the people. Presently a miUtary dictator, Bibik 
Maloof, rules Ngolo and in Esteban the chief of the nunority tribe is in power. In the past the 
ancestors of the people of Ngolo and Esteban continually warred with the ancestors of Alantra. 
These wars, although fought as rebgious wars, were in fact over water rights, water transportation, 
and access to seasonal hunting areas in the marshlands around Lake Sharino. This area of Lake 
Sharino, known as “The Font” has been in dispute for over 2000 years. "The Font" is now a rich 
agricultural plain and also contains the northern dam, that controls the water flow from Lake 
Sharino. No natural boundary or separation exists between Alantra and Ngolo nor between Alantra 
and Esteban. 

In the past, Alantra, Beyer, and Chardeaux were dependent upon fishing, husbandry and agriculture 
for prosperity. Today, all three of the nations are prosperous and have bght industry, a well- 
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developed agricultural capability, and a limited fishing industry. The three nations have long shared 
the major water resource, Lake Sharino. Two years ago they signed the second 100 year co¬ 
prosperity treaty that addresses the lake, mutual trading, and identifies other favored trading 
partners. 

Ngolo and Esteban historically have been hunters and gatherers. Today, Esteban is barely above the 
poverty level, has no industry, and only a limited agricultural base. Ngolo, on the other hand, has a 
fledgling light industry base, only a limited agricultural base, and all property is state owned and 
controlled. All power and wealth are tightly held by the ruling and military class. Ngolo could 
increase its prosperity if agrarian and land reform policies were enacted. However, to date the 
ruling dictator has not made any attempt at reform. 

Alantra, Beyer, and Chardeaux have self-defense forces that operate under control of the civil 
government. Alantra forces are fairly modem and include: a) land forces, consisting of one 
motorized rifle division, two reconnaissance battalions, and one tank brigade; b) air forces, 
consisting of one helicopter transport squadron, local air defense sites (located along the adjacent 
borders with the countries of Ngolo and Esteban and around the two dams on Lake Sharino, and 3 
medium lift fixed-wing transports; c) water-borne forces, consisting of two special purpose 
(reconnaissance) platoons, two coastal water craft squadrons, and the lake-district surveillance 
helicopters. Beyer and Chardeaux have similar capabiUties, although less modem forces. 

Very little is known about the disposition of military forces in Ngolo and Esteban. Recently Ngolo 
has attempted to modernize and strengthen their military forces. It is thought that some mechanized 
capability has been developed. Ngolo has one under-strength motorized rifle division, possibly one 
mechanized rifle brigade and two tank battalions. The elite force is the Royal Guard, a brigade size 
force with modem arms and assault vehicles. They have recently added point air defense 
capabilities and it is believed they have added a limited attack helicopter capability. In addition, 
some older light transport hehcopters are available and the only fixed-wing assets are civil air; 
however, these could be co-opted by the military. Esteban’s forces are likely less modem than 
Ngolo’s. Both countries have large “bandit” forces that are organized around local leaders and are 
at times very active. These bandit forces are well armed, with individual weapons, and well 
organized. 

Recent History: 

1 year ago 

A three-year long drought continues. Alantra, Beyer, and Chardeaux are managing to remain 
reasonably self-sufficient. This is due to the enlightened management of the water resources of 
Lake Sharino, and the economic and trade alliances among themselves and other nations outside the 
region. Conversely, Ngolo and Esteban are fast approaching poverty and mass hunger. The rivers in 
the region depend upon the seasonal rains, which have been absent for three years. They are totally 
dependent upon the good will of Alantra and Chardeaux for water for the crops necessary to feed the 
people. For the last two years, Alantra, Beyer, and Chardeaux also provided approximately 50% of 
the required foodstuffs to Ngolo and Esteban. 
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6 months ago 


Alantra, Beyer, and Chardeaux amended the 100 year co-prosperity treaty to remove Ngolo and 
Esteban as favored trading partners. Further, they reduced the amount of water that could be taken 
from the lake by 20% for each of the three countries. As a result of these new restrictions, Alantra 
reduced the flow of water to Ngolo and Esteban by 80%. Chardeaux soon followed and completely 
shut-off water sources for Ngolo and Esteban. Beyer discontinued all aid and trade with Ngolo and 
Esteban. Alantra soon followed. 

Currently: 

Alantra, Beyer, and Chardeaux are now suffering the effects of the drought and resultant famine, 
although not to the same extent as Ngolo and Esteban. Alantra, Beyer, and Chardeaux have ended 
all trade and aid to Ngolo and Esteban. Additionally Alantra announced that they would deny all 
water resources soon. Esteban has turned to their neighbor to the South for relief. This neighbor 
has provided some water and some foodstuffs, but can only do this for a short time. Ngolo has 
found itself completely cut-off from all water sources. The drought, lack of water sources, poor 
agrarian policies, and a history of social injustice have isolated Ngolo from its neighbors. Only 
Esteban has any semblance of normal relations with Ngolo. Starvation, bandit activity, popular 
uprisings, military coup, or attack on neighboring nations are all possibihties for Ngolo. 

Outlook: 

Although the situation is dire in Ngolo and the pohtical and miUtary situation is tense, currently the 
citizens appear to be conducting business as normal. That is, they continue to barter and trade with 
Esteban and petition the other nations in the area for increased support and aid. 

Additionally, there are some indications that Ngolo dictator Malouf was advised by his brother, the 
eunbassador to Chardeaux, to conduct war games in order to bolster national pride and to show its 
military power and will to neighboring nations and the minority and dissident factions within the 
country. 

As the situation continues to worsen, the only rallying point left for Malouf may be the ancient 
battle cry of “The Font, the Font, on to the Font”. If he chooses this option, it is anticipated that 
Ngolo will invade Alantra to seize control of the southern dam in Alantra and the rich agricultural 
fields in the area, thereby meiking water and foodstuffs available to Ngolo. 
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Information Sources: 


Analysts within the inteUigence directorate rely on three source groups: 

■ Source Group A: Open Sources 

The sources in this group are "open" sources. That is newspapers, magazines, national/intemational 
television and radio, and international organizations. The information from these sources can 
normally be taken at face value. These sources normally report activities and only rarely add a 
biased interpretation. 

■ Source Group B: Established Covert Sources 

The information sources within this group are highly placed within various agencies and 
organizations of the countries of interest. The information provided is normally very accurate. 

■ Source Group C: New Covert Sources 

The information sources within this group have been more recently recruited than those in Group B. 
At times, the information provided is not accurate. There is no concern of compromise and the 
inaccuracies are probably as a result of inexperience. 
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VERBAL INSTRUCTIONS FOR EXPERIMENT (A/A) 


Welcome and Purpose of Study. 

Thank you for coming to participate in our study. The purpose of this study is to investigate if 
the way in which information is presented affects how people use that information to make 
decisions. In this study, you will be acting as an analyst in a government command center in 
the imaginary country of Alantra. You have recently been assigned as a senior analyst within 
the intelligence directorate. You are supporting the Alantra political leadership and various 
Alantra embassy officers throughout the region. You are being supported by another, more 
junior, analyst within the directorate. Your reports and analyses will be provided to your 
supervisor, the Analyst Supervisor. 

Show the organization chart (blue paper) 

Note: the participant will receive a hard copy of the org chart with the 

"Background" handout 


Alantra Intelligence Directorate 
Organization Chart 
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You will monitor a series of incoming messages pertaining to a particular fictional 
geographic region in order to estimate the likelihood of a future event in the country of 
Ngolo. In addition, from time to time during the scenario you may receive requests for 
various types of assessments from Alantra embassy staff in the surrounding countries and 
from the Chief of the Alantra Intelligence Directorate. Your position acts as the liaison for 
these requests with the other members of your analyst unit, which includes the analyst 
supervisor and the junior analyst in addition to yourself. When requests of this type are 
received, all members of the unit are invited to provide input. This helps to ensure that all 
staff with relevant knowledge regarding an issue are included. 

The study will be conducted on a computer and will ask you to make assessments 
regarding messages pertaining to an imaginary situation. I will explain how to make these 
assessments and respond to the requests from embassy staffs and the Chief of the 
Intelligence Directorate. I will also walk you through a practice session so that you can 
become more comfortable with the simulation. 

Your responses will be used by SPAWAR Systems Center for research purposes only. 
Your individual responses will be kept anonymous and will only be released when 
combined with those of other participants. Please feel free to ask questions at any time. 
Your participation is voluntary and you may terminate the study at any time with no 
adverse consequences. In addition to evaluating messages, we also ask you to complete a 
short background survey. 

Consent Form and Demographics/Cultural Values Survey. 

=» Hand out the survey and informed consent sheet. Collect them when the 

PARTICIPANT IS FINISHED. 

At this point, I would like you to read and fill out a consent form if you agree to 
participate, and then to ffli out a background survey. 

Political Background and Current Situation Handout. 

Hand out “Setting & Tasks, Background, Recent History, Current Situation & 
Outlook” and a copy of the Map, 

Please read this operational scenario carefuliy. It provide information about the political 
background and current situation in the imaginary countries of Alantra, Ngolo, Esteban, 
Beyer, and Chardeux. Do not try to memorize this information; you can refer to these 
papers throughout the scenario. Your task, as a government command analyst of Alantra, 
will be to monitor and evaluate message traffic pertaining to possible actions by Ngolo. 

While participant is reading over the scenario, the experimenter will review the 
participant’s background survey and consent form to insure that they are 
COMPLETE. The experimenter will also use this time to configure the computer for 
THE TEST conditions. 
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Training. 


==» NOTE: The experimenter advances through the screens and demonstrates how to 

MAKE THE ASSESSMENTS DURING THE FIRST SIX (6) MESSAGES OF THE PRACTICE SET. THE 
INITIAL SCREEN CONTAINS INSTRUCTIONS FOR THE PARTICIPANT. 

Please read the screen entitled “What To Do.” This screen describes your tasks and the 
event you will be considering. Please let me know if you have any questions. 

=> Areas for the experimenter to emphasize: 

1. The event category of interest — a possible attack on Alantra. 

2. The periodic assessments (and conHdence ratings) after a block of messages. 

3. Marking of 1-5 messages most important in making the assessment. 

4. The fact that, during practice , the participant will evaluate messages pertaining a 
possible attack by Beyer. Stress that, during practice and the “real” scenario, the 
participant is acting as an analyst of Alantra. (The participant is always an analyst 
of Alantra—during practice he/she evaluates messages pertaining to likelihood of 
attack by Beyer, and during the “real” scenario,” he/she evaluates messages 
pertaining to the likelihood of attack by Ngolo.) 

=» The experimenter will advance the screen to the practice session, and will 

DESCRIBE THE SCREEN DISPLAY TO THE PARTICIPANT: 

The first screen requests an initial assessment of what likelihood you would assign to the 
probability of Attack based on your reading of the background information. Following 
each assessment of this type, you will be asked to provide a conHdence rating for your 
assessment. 

^ THE EXPERIMENTER WILL ADVANCE THE SCREEN TO THE FIRST PRACTICE MESSAGE. 

The next computer screen contains a map of Ngolo, Esteban, Alantra, Beyer, and 
Chardeux, as well as the locations of the avenues of transportation (railroads, bridges, and 
airports), troop deployments, and message sources. The sources for messages include 

■ "Open" sources (green background). These are newspapers, magazines, 
national/international television and radio, and international organizations. The 
information from these sources can normally be taken at face value. These sources 
normally report activities and only rarely add a biased interpretation. 

■ Established covert sources (blue background). The information sources within this group 
are highly placed within various agencies and organizations of the countries of interest. 
The information provided is normaliy very accurate. 

■ New covert sources (yellow background). The information sources within this group 
have been more recently recruited than those in the Established Covert Sources group. 
At times the information provided is not accurate. There is no concern of compromise 
and the inaccuracies are probably as a result of inexperience. 
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The screen also shows the day and time the message was sent, the elapsed time during the 
scenario (faster than real time), and the time since the last message. 

For each message, there will be a flashing circle on the screen that will correspond to the 
area that the message describes. Some messages pertain to Ngolo's mechanized infantry, 
engineers, reconnaissance, and helicopter landing areas—the location of these sites is also 
included on the map and their meaning is listed on a key to the right of the screen. 

=» THE EXPERIMENTER WILL ADVANCE THE SCREEN THROUGH 6 MESSAGES TO AN ASSESSMENT 
BLOCK, AND WILL DEMONSTRATE MAJONG AN ASSSESSMENT, A CONFIDENCE RATING, AND 
MARKING IMPORTANT MESSAGES. 

During the practice session, you will be making assessments of the likelihood of attack by 
Beyer, and you will be rating your confldence in your assessments. When you are asked 
for this information, the upper left screen will list the previous messages since the last 
assessment. This is like the initial assessment and confldence ratings except there were no 
previous messages. You will also be asked to mark the messages (up to 5) that were most 
important in making your assessment. To mark a message, just click anywhere on the line 
of the message. 

Practice Session. 

THE PARTICIPANTS "TAKES OVER THE CONTROLS." WHEN THE FIRST E-MAIL MESSAGE IS 
RECEIVED, THE EXPERIMENTER WILL TALK THE PARTICIPANT THROUGH IT. 

This screen shows a request from an embassy for an evaluation of some situation. (Read 
through the request.) Remember that you are the liaison for these requests with the other 
members of your analyst unit, which includes the analyst supervisor and the junior 
analyst in addition to yourself. All members of the unit are invited to provide input. Again, 
this helps to ensure that all staff with relevant knowledge regarding an issue are included. 
The procedure for replying to these requests is as follows: 

1. Read the request and make your initial evaluation. 

2. Forward the request and initial evaluation to the group (your supervisor and the 
junior analyst) for their review and input. 

3. When the responses from your supervisor and the technician are received, prepare a 
final evaluation. 

Note : Although all staff in the analyst unit are invited to provide input, the final 
evaluation is your choice. 

4. Send the final evaluation directly to the requestor. 

=> THE PARTICIPANTS PROCEEDS THROUGH THE FINAL SIX (6) MESSAGES IN THE PRACTICE SET, 
ENDING AT A TRANSHION SCREEN. 







Instructions Summary. 


=» The experimenter ascertains that, following practice, the screen is advanced to 

THE “FINAL INSTRUCTIONS.” 

Please review the instructions and let me know if you have any questions. 

=» Areas for the experimenter to emphasize: 

1. The event of interest. 

2. The periodic assessments (and confidence ratings) after a block of messages. 

3. The need to mark 1-5 messages most important to each assessment period. 

4. The procedure for responding to embassy and Chief of Intelligence Directorate 
requests. 

5. The fact that the participant will now evaluate messages pertaining to the possibility of 
an attack by Ngolo. Stress that, as during practice, the participant is acting as an 
analyst of Alantra. 

6. The fact that the experiment is self-paced. 

(On-screen versions of scenario information: 

“NEXT” button sends to update on screen, 

“BACKGROUND” button sends to full background; suggested only if participant is confused 
or wants to review information—however, they have hard copies of all.) 


Update to Current Situation; "Ngolo Update" 

=> Advance the screen. On the final screen before beginning the scenario, the 
experimenter should emphasize the Ngolo Update: Previous Watchstander’s 

REPORT: 

Before beginning the study, please review the update on the situation in Ngolo. This is 
previous watchstander's analysis of the situation, and is the most up-to-date information 
available regarding the situation. 

Allow partieipant to review the previous watchstander's assessment. 

Study Scenario (Part 1). 

Please work at your own pace and respond as you did during practice to the assessment 
blocks and requests. Expect to take about a half-hour to complete the flrst 50 messages. 
The study will start by asking you for an assessment based on the current situation in 
Ngolo. Following this, there will be 50 messages divided into blocks of 10 — after each 
block you will be asked by the Alantra government headquarters for your assessment and 
for a rating of the confldence of your assessment (remember tbat during tbe assessment. 
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the screen will list the previous 10 messages to assist in your decision). Please also mark 
those messages most important in making your assessments, 1-5 per block of 10. In 
addition, you may receive e-mail requests for various types of evaluations from embassy 
staffs and the Chief of the Intelligence Directorate for After completing the first 50 
messages you may take a 5-10 minute break before starting the last 50 messages. 

The experimenter advances the screen to the initial “real” assessment. 

(“BACK” button sends back to final instructions screen, 

"START” button sends initial assessment.) 

The experimenter observes the participant(s) and assists with problems. The 

EXPERIMENTER SHOULD ONLY REITERATE AND CLARIFY PREVIOUS INSTRUCTIONS; THE 
EXPERIMENTER SHOULD NOT ATTEMPT TO ASSIST THE PARTICIPANT WITH INTERPRETATION OF 
MESSAGES. 

Break 

You may take a 5 -10 minute break if you wish, before starting the remaining 50 
messages. 

Study Scenario (Part 11). 


Please follow the same format that was utilized for the first 50 messages. Are there any 
questions? Please begin. 


Structured Interview. 

For the last part of this study 1 would like to ask you some questions about the scenario: 

1) How important to your assessments of the likelihood of attack was the source of the 
messages? 

Follow-up probes: 

What did you consider with regard to the source? 

Which source did you trust the most? The least? 

How consistent were the sources as to whether or not attack seemed likely? 

2) How did you choose the messages that were most important to your assessments? 

3) What had the biggest impact on your confidence ratings? 
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4) With regard to requests from embassy staffs and Chief of Intelligence Directorate: 
How often did you change your initial evaluations after receiving feedback from your 
team? 

What was your strategy? 

5) Was there some point in the scenario when you reached a flnal opinion regarding the 
likelihood of attack? 

6) Was there any information not provided that you would liked to have had to help you 
evaluate the scenario? 

7) Please give us any other comments you have about the study. 

Conclusion. 


Thank you for your participation. Do you have any questions about this study? 

=> After the participant has departed, the experimenter well copy the appropriate 

EXPERIMENT LOG FILE ONTO A FLOPPY DISK FOR DATA ANALYSIS. 
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STEP-BY-STEP PROCEDURES 


1) Welcome and Purpose of Study - Thank subject for participation; present overview of 
study and show Organization Chart (blue paper); offer to answer any questions; discuss 
confidentiality. 

2) Consent Form and Demographic/Cultural Values Survey - Hand out a Consent Form 
and the Demographic/Cultural Values survey for each participant to fill out. 

3) Political Background and Current Situation Handout - Hand out a "Setting & Tasks, 
Background Current Situation, & Outlook" sheet and a copy of the map to each 
participant. (While participant is reading over the material, the experimenter looks over 
the Consent Form and the Background Survey to make sure that they were answered 
thoroughly and correctly). 

4) Training and Practice Scenarios - Familiarize participant with map (location of countries, 
sources of messages, transportation sites, troop deployments, and meaning of flashing 
circles); familiarize participant with protocol for responding at assessment blocks and to 
requests. 

Monitor messages and assess likelihood of an attack by Beyer with participant to help 
insure that participant feels comfortable with exercise. In Practice Set 1 (f 6 messages), 
the experimenter guides the participant. In Practice Set 2 (2"“* 6 messages), the participant 
manipulates the display on his/her own. 

Assist participant in responding to requests of embassy staffs and Chief of Intelligence 
Directorate. 

(Transition to beginning of experimental scenario) 

6) Instructions Summary - Briefly review with participant the instructions for progressing 
through the scenario. 

7) Ngolo Update: Previous Watchstander's Assessment - On the next screen, direct the 
participant to the Ngolo Update. Emphasize that this is previous watchstander's analysis 

of the situation, and is the most up-to-date information available regarding the situation. 
Allow participant to review the previous watchstander's assessment. 

8) Study Scenario (Part II - Participant makes initial assessment of events probability, 
responds to first set of 50 messages and provides assessments of event probability and 
ratings of confidence of the assessments (in blocks of 10 messages). 

9) Break - 5 - 10 minute break. 
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10) Study Scenario (Part III - Participant responds to second set of 50 messages and provides 
assessments of event probability and ratings of confidence of the assessments (in blocks 
of 10 messages) 

11) Structured Interview - Ask participant structured interview questions regarding the 
scenario and their hypotheses. 

12) Conclusion - Thank the participant and answer questions he/she may have. 
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FINAL INSTRUCTIONS 


Vour task ts to monitor and evaluate message traffic pertaining to Ngolo 

As you saw during the practice, you will see each message in the order it 
was received (but not In real time). Each message includes: 

(1} the message text, 

(2) the message source, 

(3) when the message arrived (e.g. Monday at 16:24), 

(4) the time (hrsrmlns) that has passed since the first message, and 

(5) the time (hrs:mlns) that has passed since the previous message. 

You must read each message to determine whether It contains evidence for or against 
one possible event: 

• ATTACK NGOLO is going to attack AI antra 

After you have finished evaluating the contents of a message, click the "NEXT* button 
to show the next message. 

Periodically, headquarters will ask you to estimate the likelihood of an attack and 
your confidence in your estimate. As in the practice session, ail the messages you 
have received since your last assessment will be displayed to help you decide how the 
situation may have changed. Please mark the messages (up to 5) that were most 
Important in making your assessment. 

You may also receive e-mail requests for Information about specific issues. The 
requests will come from Alantra embassy officials in the surrounding countries or 
from the Chief of the Alantra intelligence Directorate. You are the liaison for this 
type of request for your analysis unit which includes your analyst supervisor and one 
junior analyst. 

When you receive one of these requests, select your Initial best answer and press 
“Share* to forward the request and your answer to your supervisor and your 
subordinate for their review. After they reply, consider their comments, retain or 
modify your initial answer, and then press “Send Answer* to return It to the embassy 
or intelligence chief. (NOTE: although the other staff are invited to provide input, the 
final answer is your choice.) 
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The following three passages were shown to participants just before they started the actual 
experiment in order to establish an anchor for the initial set of messages. Each participant 
viewed only one of these passages, depending on the anchor condition to which he had been 
assigned. They were presented in the context of being the latest analysis and estimate of 
situation of the previous analyst. 

War Games Anchor: 




NGOLO UPDATE 


Previous Watchstander's Analysis and Estimate of Situation 


Ngolo has the capability to conduct vv'argames or to conduct an attack against Alantra. Based on 
recent events and historical tendencies it is very likely that Ngolo will soon announce 
wargames and then conduct them on Ngolon solL 

The purpose of the VY'argarnes will be to bolster national pride and to shovv* military povv^er to 
neighboring nations and the minority and dissident factions within Ngolo. 

Anticipate wargame exercises, which may appear to be preparation for attack, to start within 
the next 96 hours. 



Attack Anchor: 


NGOLO UPDATE 

Previous Watchstander’s Analysis and Estimate of Situation: 

Ngolo has the capability to conduct wargames or to conduct an attack against Alantra. Based on 
recent events, the worsening situation, and historical tendencies it is very likely that Ngolo 
will soon invade Alantra. 

Ngolo's purpose '/('llI be to seize control of the southern darn in Alantra and the rich agricultural 
fields in the area, thereby making water and foodstuffs available to Ngolo. 

Anticipate an attack, possibly under the guise of wargame exercises, within the next 96 hours. 
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Neutral Statement; 


NGOLO UPDATE 

Previous Watchstander*s Analysis and Estimate of Situation: 

Ngalo has the capability to conduct v/argames or to conduct an attack against Alantra. 

If Ngoio attacks Aiantra they v/ill acquire the rich agricultural fields in the area, thereby 
making water and foodstuffs available to Ngoio. 

If Ngoio engages in y/argarne eKercises,they wi I I do so to bolster national pride and to show 
military power to neighboring nations and the minority and dissident factions within Ngoio. 

One or the other is expected to start vyithin the nest 96 hours. 
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